Skip to content

fix(docker): pin pip, uv and certifi to reject the eMudhra chains - #239

Merged
lvrach merged 2 commits into
mainfrom
chore/sec-707-emudhra-pins
Oct 27, 2026
Merged

fix(docker): pin pip, uv and certifi to reject the eMudhra chains#239
lvrach merged 2 commits into
mainfrom
chore/sec-707-emudhra-pins

Conversation

@lvrach

@lvrach lvrach commented Oct 22, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

What changed

The Profiles IDE image now pins the TLS trust for Python tooling:

  • apt-get upgrade and an explicit ca-certificates install take Ubuntu's 20260816ubuntu0.22.04.1 store, which drops the emSign roots.
  • pip>=26.2.2 and certifi>=2026.10.23 replace the plain pip3 install --upgrade pip.
  • uv 0.13.4 is copied from ghcr.io/astral-sh/uv by index digest into /usr/local/bin. setup.sh finds it and skips its unpinned astral.sh install. uv keeps its bundled roots, so its trust does not depend on the apt layer.
  • uv lock --upgrade-package 'certifi>=2026.10.23' raises certifi in the profiles-mcp v1.4.0 lock before setup.sh, so uv sync and uv run in start.sh keep it.

Why

The IDE runs pip and uv at run time, and the bundled profiles-mcp server resolves mcp[cli] from PyPI on every start. The current image trusts the compromised eMudhra intermediate chains in pip 26.1.2, uv 0.11.26, certifi 2025.04.26 (profiles-mcp venv) and the Ubuntu CA store. Ticket: SEC-707 (parent SEC-607).

How to verify

The image was verified locally with the probe harness on a Mac mini. The build used the same base digest and the same apt, pip, uv and profiles-mcp steps as this Dockerfile (arm64). Results:

  • versions: ca-certificates 20260816ubuntu0.22.04.1, pip 26.2.2, uv 0.13.4, certifi 2026.10.19 in the system Python and in the profiles-mcp .venv;
  • pip and uv fail TLS against both eMudhra test hosts and resolve from pypi.org;
  • stdlib ssl and certifi reject the test hosts and reach a normal host;
  • uv run --with "mcp[cli]" in the profiles-mcp project starts and keeps certifi 2026.10.19.

The Dockerfile carries pins only. It has no build-time version checks.

Release path

This is a fix: commit, so release-please opens the next patch release PR. Publishing that release runs build-docker.yaml, which pushes profiles-code-server:v<version> to ECR. Then the rudder:release-profiles-ide-infra flow bumps RSOURCES_EXT_WHT_CODESERVERIMAGENAME in rudderstack-operator operator-helm/valuefiles/values.sources.yaml (prod runs v1.20.3 today) and the image warmers in rudder-devops helm-charts/cluster-overprovisioner/values.*.yaml (v1.20.2 today).

Not covered

  • The image runs Python 3.10, which is end of life. Moving to a supported Python is a follow-up.
  • The certifi bump in the profiles-mcp repo lock. The image raises it at build time instead.
  • The full image (code-server .deb, Cline extension, pb-proxy) was not built locally. CI builds it on both architectures.

certifi 2026.10.23 also drops the emSign C1 and C3 roots. The pin skips the 7-day Dependabot cooldown because this is a security fix.

🤖 Generated with Claude Code

@lvrach
fix(docker): pin pip, uv and certifi to reject the eMudhra chains
b46f057 
The Profiles IDE runs pip and uv at run time, and the bundled
profiles-mcp server resolves packages from PyPI on every start. The
current image trusts the compromised eMudhra chains in four places:
pip 26.1.2, the unpinned uv 0.11.26 from the astral.sh install script,
certifi 2025.04.26 in the profiles-mcp lock, and the Ubuntu CA store.

- apt-get upgrade takes ca-certificates 20260816, which drops emSign.
- pip 26.2.2 and certifi 2026.10.19 replace the plain pip upgrade.
- uv 0.13.4 comes from its image by digest, so setup.sh skips its own
  install. uv keeps its bundled roots, which reject the chains,
  so its trust does not depend on the apt layer.
- uv lock raises certifi in the profiles-mcp lock before setup.sh.

Verify: in the built image, pip and uv fail TLS on an emSign test host
and reach pypi.org; certifi is 2026.10.19 in both Pythons.

Refs: SEC-707

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

🔒 Scanned for secrets using gitleaks 8.30.1
@lvrach
lvrach requested a balanced review from Copilot October 22, 2026 14:13
Copilot AI reviewed Oct 22, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Security-sensitive trust changes need human confirmation of full-image builds and certificate-rejection behavior on both supported architectures.

No open issues 🎉

What changed in this PR

Updates the Profiles IDE image’s Python tooling and certificate trust to reject eMudhra chains, including the bundled profiles-mcp environment.

Changes:

  • Upgrades Ubuntu packages and explicitly installs ca-certificates.
  • Sets minimum pip and certifi versions and copies uv 0.13.4 from a digest-pinned image.
  • Updates certifi in the profiles-mcp lockfile before setup so subsequent syncs retain the updated version.
File Description
Dockerfile Updates system and Python trust dependencies and the profiles-mcp lockfile.

Review effort: Balanced

@github-actions

github-actions Bot commented Oct 22, 2026

Copy link
Copy Markdown
Contributor

Docker Preview Images Ready

Preview Docker images have been built for this PR:

ECR (internal):

docker pull 422074288268.dkr.ecr.us-east-1.amazonaws.com/rudderstack/profiles-code-server:pr-239

Image Details:

  • Tag: pr-239
  • Platforms: linux/amd64, linux/arm64

Test the image:

docker run --rm -p 8080:8080 422074288268.dkr.ecr.us-east-1.amazonaws.com/rudderstack/profiles-code-server:pr-239

Note: These preview images will be overwritten on subsequent pushes to this PR.

@lvrach
fix(deps): bump certifi to 2026.10.23 to drop emSign roots [SEC-771]
c769050 
certifi 2026.10.23 also drops the emSign C1 and C3 roots that 2026.10.19
restored. Verify: the pinned version reads 2026.10.23 and the lock
resolves to it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

🔒 Scanned for secrets using gitleaks 8.30.1
@lvrach

lvrach commented Oct 26, 2026

Copy link
Copy Markdown
Member Author

Security-sensitive trust changes need human confirmation of full-image builds and certificate-rejection behavior on both supported architectures.

Full-image builds pass on both architectures. Build and Release run 40095935809 on c769050 passed docker-build-ecr (ubuntu-latest, amd64), docker-build-ecr (arc-arm64, arm64) and docker-manifest-ecr. The rejection probe ran on arm64 only. The pinned parts do not depend on the architecture: ca-certificates is an all package, pip and certifi are pure-Python wheels, and the uv copy uses one multi-arch index digest.

@lvrach
lvrach merged commit 40199ee into main Oct 27, 2026
28 checks passed
@rudderstack-github-actions rudderstack-github-actions Bot mentioned this pull request Oct 27, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@vamsikrishnakandi vamsikrishnakandi vamsikrishnakandi approved these changes
@kuldeep0020 kuldeep0020 kuldeep0020 approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@lvrach @vamsikrishnakandi @kuldeep0020